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(54) Tide: AN APPARATUS AND METHOD TO GUARANTEE FORWARD PROGRESS IN A MULTITHREADED PROCESSOR 



(57) Abstract 

A system and method for performing computer process- 
ing oprearJons in a data processing system includes a multi- 
threaded processor (100) and thread switch logic (400). The 
multithreaded processor is capable of switching between two 
or more threads of infttmrtinr™ which can be i n de penden tly 
executed. Each thread has a corresponding state in a thread 



^^toftnare 



state register (440) depending on its execution status. The 
thread switch logic contains a thread switch control register 
(410) to store the conditions upon which a thread will occur. 
The thread switch logic has a time-out register (430) which 
forces a thread switch when execution of the active thread 
in the multithreaded processor exceeds a programmable pe- 
riod of time. Thread switch logic also has a forward progress 
count register (420) to prevent repetitive thread switching be- 
tween threads in the multithreaded processor. Thread switch 
logic also is responsive to a software manager (460) capable 
of changing the priority of the different threads and thus su- 
perseding thread switch events. 
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Description 

An Apparatus and Method to Guarantee Forward Progress in a 

Multithreaded Processor 

Related Application Data ; , . 

5 The present invention relates to the following U.S. patent 

, -applications the subject matter of which is hereby 
incorporated; by reference: (1) U.S. application entitled 
Method? and Apparatus for Selecting Thread Switch Events in a 
Multithreaded Processor, Serial Number 08/958,716 filed 23 

10 October 1997 , filed concurrently herewith; (2) U.S. application 
entitled Thread Switch Control in a Multithreaded Processor 
System, Serial Number 08/957,002 filed 23 October 1997 
concurrently herewith; (3) U.S. application entitled Altering 
Thread Priorities in a Multithreaded Processor, Serial Number 

15 08/958,718, filed 23 October 1997, filed concurrently herewith; 

(4) U.S. application, entitled Method and Apparatus to Force a 
Thread Switch in a Multithreaded Processor, Serial Number 
08/956,577 filed 23 October 1997, filed concurrently herewith; 

(5) . U„S« application entitled Background Completion of 
20 Instruction and Associated Fetch Request in a Multi thread 

^Processor, Serial Nu^er ^773,572 filed 27 December 1996; (6) 
- U.S. application, entitled Multi-Entry Fully . Associative 
Transition cache, Serial .Number 761.378 filed 09 December 1996; 
(7) U.S. application, entitled- Method and Apparatus for 
25 . . Prioritizing and Routing Commands, from a Command Source to a 
Command Sink, Serial Number 761, 380 filed 09 December 1996; (8) 
U.S. application entitled Method and Apparatus for Tracking 
Processing of a Command f Serial Number 761,379 filed 09 
: December 1996; (9) U.S. application entitled Method and, System 
30 for Enhanced Multi thread Operation in a Data Processing System 

by Reducing Memory Access Latency Delays, Serial Number 473 , 692 
, filed 7 June 1995; and (10) U.S. Patent 5,778,243 entitled 
Multithreaded Cell for a Memory, issued. 07 July 1998.. 
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Background of the Invention 

The present invention relates in general to an i m p ro ved 
method for and "apparatus L of a* computer data processing system; 
and in particular,' to an ~ . improved high performance 
multithreaded computer data processing system and method 
embodied in the hardware of the '"processor. 

1 The fundamental structure of a modern computer includes 
peripheral devices to communicate Unfonriation to Szid-from the 
outside world; such peripheral devices . may be keyboards, 
monitors, tape drives, communiciti6n : liheis coupled to a 
network; etc. Also included in the basic structure of the 
computer is the hardware necessary to receive, process, and 
deliver this information from and to the outside world, 
including busses, memory units, input/output (I/O) controllers, 
storage devices, and at least one central processing unit 
(CPU), etc*.' The CPU is the brain of the system. It executes 
the instructions which comprise a computer program and directs 
the operation of the other system components. 

From the standpoint of the computer's hardware, most 
systems operate in fundamentally the same manner. Processors 
actually perform very simple operations Quickly, such as 
arithmetic, logical comparisons, and movement of data from one 
ideation to another. Programs which -direct a computer to 
perform massive numbers- of * these simple operations give the 
illusion that the computer "is doing something sophisticated. 
What is perceived by ; tJie ^ser as a new or improved capability 
of a computer system, ' however, may- actually be the machine 
performing the same simple operations , r but ; much faster. 
Therefore ' continuing improvements 7 to computer systems require 
that these systems be made ^ever -faster. 

" One measurement of the overall speed of a computer system, 
also called the throughput^ is r measured as the number of 
operations performed per unit of time. Conceptually, the 
simplest of all possible improvements to system speed is to 
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increase the clock speeds of the various components, 
particularly the clock;, speed of the processor. So that if 
everything runs twice as fast but otherwise works in exactly 
the same manner, the system will perform a given task in half 
5 the time. Computer processors which were constructed from 

discrete components years ago, performed significantly faster 
- by shrinking the size and reducing the number of components; 
•. -;^eyfi»tually the entire processor was packaged as an integrated 
circuit on a single chip . The reduced size made it possible 
10 . to increase the clock speed of the processor, and accordingly 
increase system .speed. 

Despite the enormous improvement in speed obtained from 
integrated - circuitry, the demand for ever faster computer 
systems still exists. Hardware designers have been able to 

15 obtain still further improvements in speed by greater 

integration, by further reducing the size of the circuits, and 
by other techniques. Designer, however, think that physical 
size reductions cannot continue indefinitely and there are 
limits to continually increasing processor clock speeds. 

20 Attention has therefore been directed to other approaches for 
further improvements in overall speed of the computer system. 

Without changing the 9199k speed, it is .still. possible to 
improve system speed by usingumultiple processors. The modest 
cost of individual processors packaged on integrated circuit 

25 chips has, made this practical* The use of slave processors 
considerably improves system speed by off-loading work from the 
CPU to the, slave processor. ; For, instance, slave processors 
routinely execute repetitive - and single special purpose 
programs, such as input/output, device communications and 

30 control. - ; It is also possible- for multiple CPUs to be placed 
in a single computer system, typically a host-based system 
which services multiple users simultaneously . Bach of the 
different CPUs can separately .execute a different task on 
behalf of a different user, thus increasing the overall speed 

35 of the system to execute multiple tasks simultaneously. It is 
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much more difficult, however, to iinprove the speed at which a 
single task, such as an application program, executes. 
■ Coordinating the execution and delivery of results of various 
functions among multiple CPUs is a tricky business. For slave 
I/O processors this is not so difficult because the functions 
are pre-defined arid limited- but for multiple CPUs executing 
general purpose application programs it is much more difficult 
: to coordinate functions because, in part, system designers do 
not know the details of the programs in advance! Most 
application programs follow a single path or flow of steps 
performed by the processor. While it is sometimes possible to 
break up this single path into multiple parallel paths , a 
universal application for doing so is still being researched. 
Generally, breaking a lengthy task into smaller tasks for 
parallel processing by multiple processors is done by* a 
software" engineer writing code on a case-by- case basis. This 
ad hoc approach is especially problematic for executing 
coiranerciai transactions which are not necessarily repetitive 
or predictable: 

Thus, while multiple processors improve overall system 
performance, there are still many reasons to improve the speed 
of the individual CPU. If the CPU clock speed is given, it is 
possible to further increase the speed of the CPU, i.e., the 
' number of operations executed per secoridv by increasing the 
avaragre " number of operations '■•"execu tied per clock cycle. A 
common architecture for -high performance, single -chip 
microprocessors is the reduced instruction set computer (RISC) 
architecture characterized by a / small simplified set of 
frequently used instructions for rapid execution, those simple 
operations performed' ? "quickly mentioned earlier^ As 
semiconductor technology has advanced, the goal of RISC 
architecture has been to develop processors capable of 
executing one or more instructions on each clock cycle of the 
machine. Another approach to increase the average number of 
operations executed per clock cycle is to modify the hardware 
withiri the CPU. This throughput measure, clock cycles per 
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instruction, is .commonly uaed to characterize architectures for 
high performance processors. Instruction pipelining and cache 
memories are computer architectural features that have made 
this achievement possible. Pipeline instruction execution 
5. allows subsequent instructions to begin execution before 
^previously issued instructions have finished. Cache memories 
S store frequently , used and other data nearer the processor atih 
,< t-^v^lQ^i^stsruction execution to continue, -in most cases, without 
: -t waiting, ■ the full access time of a main memory. Some 
10 3 i^nprpyement has: also been demonstrated with multiple execution , 
units, with- look ahead hardware for finding instructions to 
execute in parallel. 

The performance of a conventional RISC processor can be 
further increased in the superscalar computer and the Very Long 

15 Instruction Word^ (VLIW) computer, both of which execute more 
than. one instruction : in parallel per processor cycle, in these 
architectures, multiple functional or execution units are 
provided to run multiple pipelines in parallel. In a 
superscalar; , architecture, instructions may be completed in- 

20 order and out-of-order. In-order completion means no 
instruction can conplete before all instructions dispatched 
- , ahead of it have been completed. Out-of-order completion means 
that an instruction is allowed to complete, before all 
instructions ahead of ;r it have been completed, as long as a 

25 predefined rules are satisfied. 

, For , both r in-prder and out-of-order execution in 
superscalar systems, pipelines will stall under certain 
circumstances. An instruction that is dependent upon the 
results of a previously dispatched instruction that has not yet 

30 , completed may cause, the pipeline to stall. For instance, 
instructions dependent on a load/store instruction in which the 
necessary data is not in the cache, i.e., a cache miss, cannot 
be executed until the, data , becomes available in the cache. 
Maintaining the. requisite data in the cache necessary for 

35 , continued execution and to sustain, a high hit ratio, i.e., the 
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number of requests for data compared to the number of times the 
data was readily available in ; tl£e" 1 cache, is not trivial 
especially for confutations involving large data structures • 
A cache miss can cause the pipelines to stall for several 
cycles, and the total amount of memory latency will be severe 
: if the data is not available? most of the time. Although memory 
devices used for main memory- ire becoming faster, the speed gap 
between such memory chips arid high- end processors is becoming 
increasingly larger • Accordingly, a -significant ' -amount of 
execution time in current high-end processor &eslgrite 'IS spent 
waiting for resolution of cache misses and' these memory access 
delays use an increasing proportion of processor execution 
time. 

And yet another technique to improve the efficiency -of 
hardware within the CPU is to divide a processing task into 
independently executable sequences * of ' instructions called 
threads. This technique is related to breaking a larger task 
into smaller tasks for independent execution by different 
processors except here the threads are 1 to **be executed by the 
same processor. When a CPU then; for any of a number.. of 
reasons, cannot continue the' processihg or execution of one: of 
these threads; the CPU switches to and executes another thread. 
This is the subject of the invention described herein which 
incorporates ' hardware - imaltithreading ,r 'to • tolerate 'memory 
latency. The term R irailtithreading M -as defined in the computer 
architecture community is not the same as the software use of 
the term which means one task subdivided into multiple related 
threads. In the architecture definition* the threads may be 
independent. Therefore "hardware multithreading w is often used 
to distinguish the two uses bf the term. The present invention 
incorporates the term nbltl threading to connote hardware 
multithreading'. ; :/ 

Multithreading permits the processors' pipeline (s) to do 
useful work on different threads when a pipeline stall 
condition is detected for the current thread. Multithreading 
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also permits processors imp lamenting non^pipeline architectures 
to do useful work for ::l a : separate thread when a stall condition 
is detected for a current thread. There are two basic forms 
of multithreading. A traditional form is to keep N threads, 
5,. or states, in the processor and . interleave the threads on a 
.cycle -by-cycle basis . This eliminates all pipeline 

dependencies because instructions in a single, thread are 
. ;? s@parate<a, ; The other form of multithreading, and the one 
considered by the present invention, is to interleave the 
10 threads on- r some long -latency event* 

Traditional forms of multithreading involves replicating 
the processor registers for each thread. Por instance, for a 
processor implementing the architecture sold under the trade 
name PowerPC™ to perform multithreading, the processor must 

15 maintain N states to run N threads. Accordingly, the following 
are replicated N times s general purpose registers, floating 
point registers, condition registers, floating point status and 
control . register, count register, link register, exception 
register, : save/restore registers, and special purpose 

20 registers. Additionally, the special buffers, such as a 
segment lookaside buffer, can be replicated or each entry can 
be tagged with the thread. number and, if not, must be flushed 
on every thread switch. ..Also, some branch prediction 
: mechanisms > e.g., the correlation register and the return 

25 stack, should also be replicated* Fortunately, there is no 

need to replicate some of the larger functions of the processor 
such as: level one instruction cache (LI I -cache) , level one 
data cache (LI D- cache) , instruction buffer, store queue, 
instruction dispatcher, functional or execution units, 

30 pipelines, translation lookaside buffer (TLB), and branch 
history .table. ^ When one, . thread encounters a delay, the 
processor rapidly switches to another thread. The execution 
of; this thread overlaps with the memory delay on the first 
- thread. 
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.*-•-- .'-•'•-••ExiBting' multithreading techniques describe switcliing 
threads on a* cache miss or a ntobry reference. A primary 
:r.:r ? example of this technique may be -reviewed in "Sparcle: An 
if- Evolutionary Design for Large -Scale Multiprocessors," by 
5 :V " 1 Agarwal et al., IEEE Micro Volume 13, No. 3, pp. 48-60, Jtine 
.wiifii .1993 # - -^as applied in a RISC architecture, multiple register 
..6 " gets' normally utilized tx> Support function calls are modified 
to maintain multiple threaLds. Eight overlapping -register 
Windows are modified to become four non- overlapping. * register 
10 sets, wherein each register set is a reserve fo±v trap and 
message handling. This system discloses a thread switch which 
■ ' ■ occurs on each first level cache miss that results in a remote 
: memory request, while this system represents an advance in the 
art, 'modern processor designs often utilize a multiple level 
15 cache or high speed memory which is attached to the processor. 
■ ' The processor system utilizes sdme n well-known algorithm to 
' decide what portion of its main 1 memory store will be loaded 
within each level of cache and- thus, each time a memory 
reference occurs which is not present within the first level 
20 of cache the processor must attempt to obtain that memory 
reference from a second or higher level- of cache. 

It is thus an object of the invention to provide ian 
improved data processing system which can reduce delays due to 
memory latency in a multilevel cache system utilizing hardware 
25 logic and registers embodied iii a mult i thread data processing 

system." 1 

Summary of the invention 

The invention addresses this object by providing a 
multithreaded processor capable of switching execution between 
30 two threads of instructions, and thread switch logic embodied 
in hardware registers with optional software override of thread 
switch conditions. Processing various states of various 
threads of instructions allows optimization of the use of the 
processor among the threads. Allowing the processor to 
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execute a second _ thread of . instructions increases processor 
utilization which ^ X$ yr otherwise idle when it is retrieving 
necessary data and/or instructions from various memory 
elements, such as caches, memories, external I/O, direct access 
storage devices for a first- thread. .The conditions of thread 
switching can be different per thread or can be changed during 
processing by the use of a software thread control manager. 

conditions of thread switching can be different per thread 
°* ^ e : changed during processing by the use of a software 
thread control manager. 

Thrashing, wherein each thread is locked in a repetitive 
cycle of switching threads without any instructions executing, 
can be prevented by the invention implementing a forward 
progress count register and method which allows up to a 
programmable ^maximum , number of thread switches, called the 
forward progress threshold after which the processor stops 
switching threads until one thread is able to execute. The 
forward progress register and its threshold monitors the number 
of thread switches that have occurred without an instruction 
having been executed and when that number is equal to a 
threshold no futher thread switching occurs until an 
instruction is executed, _The .forward progress count register 
can be customised, for v certain latenacy events, such as one 
: threshold value for a very long latency event such as access 
to external computer networks; and another forward progress 
threshold for shorter latency r events such as cache misses* In 
these cases, the processor system may use a software 
instruction and the. hardware response to an external processor 
latency event to block the incrementing of the forward progress 
counter in, the -thread state register, in other words, to block 
the change of ,,ftatja pf _ the particular thread in the thread 
state register. Yet, long latency events can also be handled 
by switching threads regardless of the state of the particular 
thread. 
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Thus/ "what has been invented is a method of computer 
processing setting a forward progress threshold value and 
counting a number of times that execution has been switched to 
a first thread without an instruction of the first thread being 
executed in a multithreaded processor. Every time a thread 
switch occurs to the first' thread, the state of that thread is 
Changed . '"Execution T is hot switched to a second thread { if the 
number of times that execution has been switched to thfe first 
thread is equal to the forward progress threshold Value'; rather 
the processor waits until the first thread is able to execute 1 
The forward progress threshold value is programmable by code 
and can be stored ih a forward progress count register. 
Typically, a latency event inhibits execution of an instruction 
in the first thread, 

' The invention is also a method of conqputer processing, 
comprising the steps of setting a forward progress threshold 
value in a forward progress count register and then resetting 
a state of a first thread in a first thread state register, 
attempting execution of the first thread in a multithreaded 
processor and changing the state ' of the first thread in the 
first thread state register. If the first thread is able to 
execute an instruction, th£h an instruction of the first thread 
executes and the state of the first thread in the first thread 
state* register is reset. Execution of another instruction in 
the first thread is attested and the state of the first thread 
again changes, if / libweV^t', the : first thread is unable to 
execute, then the method deterrnines if the state of the first 
thread is equal to a ^ fbft^rd ptogress threshold value. If the 
state of the first ttofead is nbt equal to the forward progress 
threshold value, the idethdd switches execution to a second 
thread in the multithreaded r proc4s^or and continues execution 
of the second thread until the second thread is u n a b le to 
execute. Execution is then* switched back the first thread in 
the multithreaded processor and the method begins anew. If, 
however, the state of the first thread is equal to the forward 
progress threshold value, then the method refrains from 
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switching execution to the second thread until the first thread 
is able to execute ^.an ^instruction to guarantee forward progress 
in the multithreaded processor having at . least two threads of 
execution. 

< The invention is simply a multithreaded processor wherein 
a thread switch is inhibited because an instruction of a thread 
'49 e ? jaot execute. ^ The. invention is also a con^uter processor 
qoc^rising a means for establishing a threshold value, a means 
for tracking, a number of thread switches in a multithreaded 
: processor and a means for inhibiting a thread switch when the 
number of thread switches equals the threshold value. 

The invention also comprises a multithreaded processor to 
switch processing between at least two threads of instructions, 
a forward progress count register operatively connected to the 
multithreaded processor and having a first forward progress 
threshold value, at least one ; thread state register operatively 
connected to the forward progress count register and to the 
multithreaded processor, to store a state of the threads of 
instructions wherein the state of each thread of instructions 
changes when the processor switches processing to each thread. 
When the state of a thread, is equal to the first forward 
progress threshold value, the multithreaded processor does not 
switch threads, but waits fpr resolution of at least one latency 
event... 

The invention also comprises a multithreaded processor 
capable of switching processing between at least two threads 
of instructions when the multithreaded processor experiences 
one of a plurality of processor latency events, a forward 
progress. count register haying at least one forward progress 
threshold value which is,,, operatively connected to the 
multithreaded processor, ( at least, pne thread state register 
operatively connected tp the forward progress count register 
and to the multithreaded .processor, to store a state of the 
threads of instructions wherein the state, of each thread of 
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instructions changes when the processor switches processing to 
each thread, and a plurality of : internal connections connecting 
the multithreaded processor tb : a 'plurality of memory elements 
wherein access to any of the plurality of memory elements by 
5 the multithreaded processor causes an internal processor 

latency event. When one of "the threads executing in the 
: t f multithreaded processor is imable^ to continue execution because 
- : of oix^ of the processor latency- events; the multithreaded 
processor switches u execution "td another "of the threads and 

10 changes the state of both threads. The state of -both threads 
are compared with at least one forward prbgfeas threshold 
value, and when the state of a particular * thread is equal to 
at least one forward progress threshold value, the 
multithreaded processor does not switch away from the 

15 particular thread. 

' The invention further comprises "at least one external 

connection connecting the multithreaded processor to an 
external memory device* a communication device* a computer 
network, or an input/output device ? wherein access to any of the 

20 devices or the network' by the multlthreatied processor causes 
an external processor latency event. An external latency event 
causing an excessive processor delay or a software instruction 
can block the change of state of the particular thread in the 
thread state register. Regardless of the state of the 

25 particular thread, the multithreaded processor still has the 
capability to switch away from the particular thread. 

Different latency events can be handled differently so the 
invention aiso coraptises j a first forward progress threshold 
value corresponding to ^ri''internal processor latency event and 

30 a second forward progress 1 threshold-valxie -corresponding to an 
external processor latency Went. Then either the 

multithreaded processor does' not- switch from a first thread if 
the first thread is &xperiebcing an internal processor latency 
event and the first forward progress threshold value equals the 

35 state 6f the first thread' or the multithreaded processor does 
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not switch from a.. second thread if the second thread is 
experiencing an external processor latency event and the second 
forward progress threshold value equals the state of the second 
thread, depending upon which thread has a higher priority. 

Other objects, features and characteristics of the present 
invention; methods, operation, and functions of the related 
elements of the structure; combinat ion of parts; and economies 
9 f ^nufacture wil^ become apparent from the following detailed 
description of the , preferred embodiments and accompanying 
drawings, all of r which form a part of this specification, 
wherein like reference numerals designate corresponding parts 
in the various figures. 

Brief Description of the Drawings 

: The, invention itself , however, as well as a preferred mode 
of use, further objectives and advantages thereof, will best 
be understood by_ reference; to the following detailed 
description, of ; an illustrative embodiment when read in 
conjunction vrith the accompanying drawings, wherein: 

Figure 1 is a block diagram of a computer system capable 
of implementing the invention described herein. 

Figure 2 illustrates. : a high level block diagram of a 
. multithreaded data processing system according to the present 
invention. . 

Figure 3 illustrates a block diagram of the storage 
control unit of Figure 2. 

Figured illustrates a block diagram of the thread switch 
logic, the storage, control unit .and, the instruction unit of 
Figure 2. 
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Figure 5 illustrate the changes of state of a thread as 
the thread experiences different chiread switch events shown in 
Figure 4- * 

Figure 6 is a flow chart of the forward progress count of 

5 a ^-* : " -the* invention. ' : " * 

' Detailed Description of the Preferred ^T^H«anfc« — 

With reference now to the figures and in particular with 
reference to Figure I, there is depicted a high levei bloc* 
diagram of a computer data processing system 10 which may be 

10 ' utilized to implement the method and system of the present 
invention. The primary hardware components and 

interconnections of a computer data processing system 10 
capable of utilizing the present invention are shown in Figure 
1 . Central processing unit (CPU) 100 for processing 

15 instructions is coupled to caches 120 i 130, and 150 . 

instruction cache 150 stores instructions for execution by CPU 
100. Data caches 120/ 130 store data to be used by CPU 100. 
The caches communicate with random access memory in main memory 
140. CPU 100 and main memory 140 also communicate via bus 

20 interface 152 with system bus 155. Various input/output 
processors (IOPs) 160-168 attach to system bus 155 and support 
communication with a variety of storage and input /output (I/O) 
devices /such as direct access stotage devices (DASD) 170, tape 
drives 172 , rembte cdnimunica tioh lines 174 , workstations 176 , 

25 and printers 178. It should be understood chat Figure 1 is 

intended to depict representative components of a computer data 
processing system 10'at'a high level, and that the number and 
types of such components may vary . 

Within " the" * CPU 100; a' processor core 110 contains 
30 specialized functional units! , each of which perform primitive 
operations, such as sequencing instructions, executing 
operations involving integers, executing operations involving 
real numbers, transferring values between addressable storage 
and logical register arrays. Figure 2 illustrates a processor 
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core 100. In a preferred embodiment, the processor core 100 
of the data processing system 10 is a single, integrated 
circuit, pipelined, superscalar microprocessor, which may be 
implemented utilizing any computer architecture such, as the 
5 family of RISC processors sold under the trade name PowerPC? 1 ; 
for example, the PowerPC 1 ? 604 microprocessor chip sold by IBM. 

As will be discussed below, the data processing system 10 
preferably includes various units, registers, buffers, 
memories, and other sections which are all preferably formed 

10 by integrated circuitry, It should be understood that, in the 
figures, ; the various data paths have been simplified; in 
reality, there are many separate and parallel data paths into 
and out of the various , components . In addition, various 
components not germane to the invention described herein have 

15 been omitted, but it is to be understood that processors 
contain additional units for additional functions. The data 
processing system 10, can operate according to reduced 
instruction set computing, RISC, techniques or other computing 
techniques. ... 

20 As represented in Figure 2, the processor core. 100 of the 

data processing, system 10 preferably includes a level one data 
cache, LI D- cache 120, a level, two L2 cache 130, a main memory 
140, and a level one instruction cache, LI I -cache 150,, all of 
which are operationally intercpnnected utilizing various bus, 

25 connections to a. storage control unit 200. As shown in Figure 

1, the storage control unit 200 includes a transition cache 210 
for interconnecting the Ll D^-cache 120 and the L2 cache 130, 
the main memory 140, and a plurality of execution units. The 
Ll D- cache 120 and Ll 1-cache 150 preferably are provided on 

30 chip as, part of the processor 100 while the main memory 140 and 

the L2 cache 130 are provided off chip. Memory system 140 is 
intended to represent random access main memory which may or 
may not be within the processor core 100 and, and other data 
buffers and caches, if any, external tq the processor core 100, 

35 and other external memory, for example, DASD 170 , tape drives 
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• 172, and workstations 176, shown is* Figure 1, The L2 cache 130 
fc is preferably a higher speed memdry^ system than the main memory 
; 140, arid by storing selected data within the L2 cache 130, the 
'memory latency which occurs as a result of a reference to the 
5* ' main memory 140 can be minimized. As shown in Figure 1, the 
; ? D2 cache 130 and the main memory 140 are directly connected to 
both the LI I -cache 150 and an instruction unit 220 via the 
- ^ ; stbrage ; ' control unit 200 

instructions from the LI I -cache 150 are preferably 1 output 

10 to dn instruction unit 220 which, in accordance with thte method 

and system of the present invention, controls the execution of 
multiple threads by the various subprocessor units, e.g*, 
branch unit 260, fixed point unit 270, storage control unit 
200, and floating point unit 280 and others as specified by the 

15 "architecture of the data processing system 10. In addition to 
the various execution units depicted within Figure 1, those 
skilled in the art will appreciate that modern superscalar 
microprocessor systems often include multiple versions of each 
such execution unit which may be added without departing from 

20 the spirit and scope of the present invention. Most of these 
units will have as an input source operand information from 
various registers such as general purpose registers GPRs 272, 
and floating point registers FPRs 28-2. Additionally, multiple 
special purpose register SPRs 274 may be utilized. As shown 

25 in Figure 2, the storage control unit 200 and the transition 

cache 210 are directly connected to : general purpose registers 
272 and the floating point registers 282: The general purpose 
registers 272 are connected to the special purpose registers 
274.- " 



30 Among the functional ^'Hardware' units unique to this 

multithreaded processor 100 Is the thread switch logic 400 and 
the transition cache 210. The thread switch logic 400 contains 
various registers that E determine which thread will be the 
active or th£ executing thread. Thread switch logic 400 is 

35 Operationally connected to the storage control unit 200, the 
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execution units 260, 270, and 280, and the instruction unit 
220. The transition cache. 210 within the storage, control unit 
200 must be . capable of implementing multithreading. 
Preferably, the storage control unit 200 and the transition 
cache 210. permit at least one. outstanding data request per 
thread. Thus, when a first thread is. suspended in response to, 
for example, the occurrence of Ll D-cache miss, a second thread 
would be able to access the LI D-cache 120 for data present 
therein. If the second thread also results in Ll D-cache miss, 
another data request will be issued and thus multiple data 
requests must be maintained within the storage control unit 200 
and the, transition, cache 210* Preferably, transition cache 210 
is the transition cache of U.S. Application Serial a Number 
08/761,378 filed 09, December 1996 entitled Multi-Entry Fully 
Associative Transition Cache, hereby incorporated by reference* 
The storage control. unit 200, the execution units 260, 270, and 
280. and the instruction unit 220 are all operationally 
connected to the thread switch logic 400 which determines which 
thread to execute. 

As illustrated in Figure 2, a bus 205 is provided between 
the storage control unit 200 and the instruction unit 220 for 
communication of, e.g., data requests to the storage control 
unit 200, and a L2 cache 130 miss to the instructipn unit 220. 
Further, a translation lookaside buffer TLB 250 is .provided 
which contains virtual- to -real address mapping. Although not 
illustrated within the present invention various additional 
high level memory mapping buf fers - may be provided such as a 
segment lookaside buff er which will operate in a manner similar 
to the translation lookaside buff er 250. 

Figure 3 illustrates ,the storage control unit 200 in 
greater detail „■ and, as the name implies, this unit controls 
the input and output of data and instructions from the various 
storage units, which include the various caches, buffers and 
main memory. As shown in Figure 3, the storage control unit 
200 includes the transition cache 210 f unctionally connected 
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to the Ll D-cache 120, multiplexer 360, the L2 cache 130, and 
main memory 140, Furthermore the transition cache 210 
receives control signals from sequencers 350, The sequencers 
350 include a plurality of sequencers, preferably three, for 
5 C handling instruction and/or data fetch requests. Sequencers 
350 also output control signais to the transition cache 210, 
the L2 cache 130, as well as receiving and transmitting control 
"signals to and from the main memory 140; ' - 

Multiplexer 360 in the storage cohtrol unit 200 shown in 
10 Figure 3 receives data from the Li b- cache 120, the transition 
1 cache 210, the L2 cache 130, main memory 140, and, if data is 
to be stored to memory, the execution units 270 and 280. Data 
from one of these sources is selected by the multiplexer 360 
and is output to the Ll D-cache 120 or the execution units in 
15 response to a selection control signal received from the 
sequencers 350. Furthermore, da shown in Figure 3, the 
sequencers 350 output a selection signal to control a second 
multiplexer 370* Based on this selection signal from the 
sequencers 350, the multiplexer 370 outputs the data from the 
20 L2 cache 130 or the main memory 140 to' the Ll l-cache 150 or 
the instruction unit 22t). In producing the above -discussed 
control and selection signals, the sequencers 350 access and 
'update the Ll directory 320 for the Ll D-cache 120 and the L2 
1 directory 330 for tiie L2 faabhe 130. 

25 with respect to the multithreading capability of the 

processor described herein, sequencers 350 of the storage 
control unit 200 also outfcmt signals to thread switch logic 400 
which indicate the stat£ of data* and instruction requests. So, 
feedback from the caches 120, 130 and 150, main memory 140, and 

30 the translation lookaside * buffer * 250 lis routed to the 
sequencers 350 and is then bommunicated to thread switch logic 
400 which may result in a thread switch, as discussed below. 
Note that' any device wherein an event designed to cause a 
thread switch in a multithreaded processor occurs will be 

35 operationally connected to sequencers 350." 
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Figure 4 is a logical representation and block diagram of 
the thread switch* logic hardware 400 that determines whether 
a thread will be switched and, if so, what thread, Storage 
control unit 200 and instruction unit 220 are interconnected 
5 with thread switch logic : 400> Thread switch logic 400 
^preferably is incorporated into the instruction unit '220 but 
if there are many threads the complexity of the thread switch 
v^logic 400 may: increase so that the logic is external . to the 
instruction^ unit 220. For ease of explanation, thread switch 
10 logic 400 is illustrated external to the instruction unit 220. 



Some events which result in a thread to be switched in 
this embodiment are communicated on lines 470 , 472, 474, 476, 
478, 480, 482, 484, and 486 from the sequencers 350 of the 

15 storage control unit 200 to the thread switch logic 400. Other 

latency events can, cause thread switching; this list is not 
intended to be inclusive; rather it is only representative of 
how, the thread switching can be implemented. A request for an 
instruction by either the first thread TO or the second thread 

20 Tl which is. not in the instruction unit 220 is an event which 

can result in a. -thread switch, noted by 470 and, 472 in Figure 
4, respectively. , Line 474 indicates when the active thread, 
whether TO or ,T1, experiences a Li p- cache 120 miss. : Cache 
misses of the L2 cache ; 130 r for either thread TO or. M is noted 

25 at lines 476 and 478>, respectively. Lines 480 and 482 are 
activated when data is returned for continued execution of the 
TO thread or for the Tl thread, respectively. Translation 
lookaside buffer misses and, completion of a table walk are 
indicated by lines 484 and 486, respectively. 

30 . .These events are all fed into the thread switch logic 400 

and more particularly to the thread state registers 440 and the 
thread switch controller 450. Thread switch logic 400 has one 
thread state register for each thread. c In the embodiment 
described herein, two threads are represented so there is a TO 

35 state register 442 for a first thread TO and a Tl state 
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- register 444 for a second thread *W1, to be described herein. 
/f -' :V -"Thread* switch logic 400 comprises- a thread switch control 
; ? ' regis iter' 410 which controls what evehts will result in a thread 
v ^v. switch* For instance, the thread switch control register 410 
5 f ■ : - can blbclt events that cause state changes from being seen by 
rJ ^ : the thread switch controller ' 450 so that a thread may not be 
;i - j: ^'BwI6etied^as a result of a blocked event- The thread switch 
' T contr6T register 410 is the subject of - U. S . application 
ehtitled Method and Apparatus for - Selecting Thread Switch 
10 'Events in a Multithreaded Processor, ^Serial Number 08/958,716 
filed 23 October 1997, filed concurrently herewith and herein 
incorporated by reference, RO997-104. The forward progress 
count register 420 is used to prevent thrashing and may be 
included in the thread switch control register 410 and is the 
15 subject of this patent application. The thread state registers 
and the logic of switching threads is the subject of U.S. 
application entitled Thread Switch Control in a Multithreaded 
Processor System, Serial Number 08/957,002 filed 23 October 
1997 concurrently herewith and --'herein incorporated by 
20 reference, RO996-042. Thread switch time-out register 430, the 
subject of U.S. application entitled Method and Apparatus to 
Force a Thread Switch in' a 'Multithreaded Processor, Serial 
Number 08/956,577 filed 23 October 1997, filed concurrently 
herewith and herein incorporated by reference, RO997-107, 
25 allocates fairness and -liveiock r issues . Also, thread 
ri priorities can be altered using software 460, the subject of 

U.S. application entitled Altering Thread Priorities in a 
Multithreaded Processor? --Serial Number 08/958,718, filed 23 
October 1997, f iled - concurrently herewith and herein 
30 incorporated by referenced RO997-106\ Finally, but not to be 
limitative, the thread switch controller 450 comprises a myriad 
of logic gates which represents- the culmination of all logic 
which actually deternEhes Whether a thread is switched, what 
thread, and under what circumstances. Each of these logic 
35 components and their functions are set forth in further detail. 
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Thread State Real at erg , 

Thread state registers 440 comprise a state register for 
each thread and, as the name suggests, store the state of the 
corresponding thread; in this case, a TO thread state register 
„ 442 and a Tl thread state register 444. The number of bits and 
, ~ the allocation of particular bits to describe the state, of each 
thread can be customised for a particular architecture and 
thread switch priority, scheme. An example of the allocation 
of bits in ~ the thread state registers 442, 444 for a 
multithreaded, processor having two threads is set forth, in the 
table belo# f 

Thread State Register Bit Allocation 

Instruction/Data 

0 = Instruction 

1 - Daia 
Miss type sequencer 

00 = None 

01 = Translation lookaside buffer miss (check bit 0 for I/D) 

10 = Lt cache miss . 

11 = L2 cache miss . 
Transition 

0 = Transition to current state does not result in thread switch 
,1 .= Transition to current state results in thread switch 
Reserved 

; • 0 • =: Load ■ 

1 =. Star? 

Reserved ' 
Forward progress counter 

111 = Reset (instruction has completed during this thread) 

000 = 1st execution of this thread w/o instruction complete 

001 = 2nd execution of this thread w/6 instruction complete 

010 = .3rd execution of tins thread w/o instruction complete 

011 = 4th execution of this thread w/o instruction complete 
100 - 5th execution of this thread w/o instruction compete 
Priority (could be set by software) 
00 = Medium 

•-'01— Low 

. 10 = High „ % 

ji = <Illeg3l> : 
Reserved-- ^y---— - 

Reserved if 64 bit implementation 

In the embodiment described herein , bit 0 identifies 
whether the miss or the reason the processor stalled execution 
is a result of a request for an instructibn or for data. Bits 
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1 and 2 indicate if the requested i^ormation was not available 
and if so, from what hardware; " '"Whether the translated 

address of the data or instruction was not in the translation 
lookaside buff er 250, or the data or instruction itself was not 
5 in the III D- cache 120 or- the* L2 cache 130/ as further explained 

in the description of Figure 5- ""Bit 3 indicates whether the 
change of state of a thread results in a thread switch. A 
z ' thread niay changd state witKb^ut 1 * rfesulfeing in a thread switch. 
For instance/ if a thread switch' occurs wh^ 1 thread Tl 

10 experiences an LI cache miss, then' ±f thread M fexperiences a 
L2 cache miss, there will be no thread switch because the 
thread already switched on a LI cache miss. The state of Tl, 
however, still changes. Alternatively, if by choice, the 
thread switch logic 400 is configured or programmed not to 

15 switch on a Ll cache miss, then when a. thread does experience 

an Ll cache miss, there will be no thread switch even though 
the thread changes state. Bit 8 of. the thread state registers 
442 and 444 is assigned to whether the information requested 
by a particular thread is to be loaded into the processor core 

20 or stored from the processor core into cache or main memory. 

Bits 15 through 17 are allocated ? tp prevent thrashing, as 
discussed later with reference to the forward progress count 
register 420. Bits 18 and 19 can be set in the hardware or 
could be set by software to indicate the priority of the 

25 thread.. ' ' ^7!^- 

Figure 5 represents foujf states 'in the present embodiment 
of a thread processed by . ; the data processing system 10 and 
these states are stored in the thread state registers 440, bit 
positions 1:2. State 00 represents the "ready" state, i.e., 

30 the thread is ready for processing because all data and 
instructions required are available; state 10 represents the 
thread state wherein the ^ecutidn of the thread within the 
processor is stalled because the thread is waiting for return 
of data into either the Ll b-cache 120 or the return of an 

35 instruction into the Ll I -cache 150; state 11 represents that 
the thread is waiting for return of data into the L2 cache 130; 
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and tlie state, 01 indicates that there is a miss on the 
translation lookaside, buffer 250, i.e. , the virtual address was 
; in error or wasn't available,, called a table walk. Also shown 
in Figure 5 is the hierarchy of thread states wherein state 00, 
5 which indicates the thread is ready for execution, has the 
.highest priority. Short latency, events are preferably assigned 
a higher priority- ; , , 

. :> ; Figure 5 also illustrates the change of states when data 
is retrieved ; from various sources. .The normal uninterrupted 

10 execution of a thread TO is represented in block 510 as state 
00. If a LI D-oache or I -cache miss occurs, the thread state 
changes to state 10, as represented in block 512, pursuant to 
a signal sent on line 474 (Figure 4) from the storage control 
unit 200 or line 470 (Figure 4) from the instruction unit 220, 

15 respectively. If the required data or instruction is in the 
L2 cache 130 and . is retrieved, then normal execution of TO 
resumes, at block 510,. Similarly block 514 of Figure 5 
represents a 1*2 cache miss which changes the state of thread 
of either TO or Tl tp .state 11 when storage control unit 200 

20 signals the miss, on lines 476 or 478 (Figure 4). When the 
instructions . pr data in the L2 cache are retrieved from main 
memory 140 and loaded into, the processor core 100 as indicated 
on lines 480, and 482 (Figure 4) , the state again changes back 
to : state 00 at block ^510. . The. storage control unit 200 

25 communicates to the thread registers 440 on line 484 (Figure 
4) when the virtual address, for requested information is not 
available in the translation lookaside buffer 250, indicated 
as block 516, as a TLB miss or. state 02. When the address does 
become available or if , there is „ a . data storage interrupt 

30 : instruction as signaled by the. storage control unit 200 on line 
. ; 486 : (Figure .4) , . the state of the thi;ea<i. then returns to state 
00, meaning ready for execution. 

t The number of states, and what each state represents is 
freely selectable by the computer architect. For instance, if 
35 a thread, has multiple LI cache , misses, such as both a LI I- 
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cache miss and LI D- cache missi ^a separate state can be 
' assigned to each type of cache miss. " Alternatively, a single 
thread state couid be assigned to represent more than one event 
or occurrence . 

5' *'•»•"•"■* k An : example of a thread switch algorithm for two -threads 
of equal priority which determines whether to switch threads 
is given. The algorithm can be expanded and modified 
accordingly for more threads and thread* switch* conditions 
according to the teachings of the invention. The interactions 

10 between the state of each thread stored in the thread state 
" registers 440 (Figure 4) and the priority of each thread by the 
thread switching algorithm are dynamically interrogated each 
J : cycle. If the active thread TO has a Ll miss, the algorithm 
will cause a thread switch to the dormant thread Tl unless the 

15' dormant thread Tl is Waiting for resolution of a L2 miss. If 
a switch did not occur and the Ll cache miss of active thread 
TO turns into a L2 cache miss, the algorithm then directs the 
processor to switch to the dormant thread Tl regardless of the 
Tl's state. If both threads are waiting for resolution of a 

20 L2 cache miss, the thread first having the L2 miss being 
resolved becomes the active thread. At every switch decision 
time/' the action taken is optimized for the most likely case, 
" resulting in the best performance. Note that thread switches 
resulting from a L2 cache miss are conditional on the state of 

25 the other thread, if hot extra thread switches would occur 
resulting in loss of performance. 

Thread Switch Control Register 

In a multithreaded processor, there are latency and 
performance penalties associated with switching threiads. This 

30 latency incliideS the time required to complete execution of the 

current thread to a point 1 where it can be interrupted and 
correctly restarted when it is next invoked, the time required 
to switch the thread- specific hardware facilities from the 
current thread's state to the new thread's state, and the time 

35 required to restart the new thread and begin its execution. 
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Prefereably the fthread- specific hardware facilities pprable 
with the inveniton inlcude the thread state registers described 
above and the memory cells described in U.S. Patent No. 
5,778,243 entitled Multithread : Cell for a Memory, herein 
5 incorporated by reference. In order to achieve optimal 

.performance in a coarse grained multithreaded data processing 
system, the t latency of an event which generates a thread switch 
^nnipi be greater, than the . performance cost associated with 
, switching threads in a multithreaded mode, as opposed to the 
10 ^normal single -threaded mode. 

^ ^The .latency of an event , used to generate a thread switch 

is dependent upon both hardware and software. For example, 
specific hardware considerations in a multithreaded processor 
: include the speed of external SRAHs used to implement an L2 

15 cache external,. to the processor chip. Fast SRAHs in the L2 
cache reduce the average latency of an LI miss while slower 
SRAMS increase the average latency of an LI miss. Thus, 
performance is gained if one thread switch event is defined as 
a Ll cache miss in hardware having an external 1*2 cache data 

20 access latency, greater than the thread switch penalty. As an 
example of how specific software code characteristics affect 
the latency of thread switch events, consider the L2 cache hit- 
to-miss ratio of the code, i.e., the number of times data is 
actually available in the L2 cache compared to the number of 

25 times data ; must be retrieved from main memory because data is 
not in the L2 cache. - A.hi^h. L2 hit-to-miss ratio reduces the 
average latency of an Ll .cache miss because the Ll cache miss 
seldom results in v a longer; ^latency L2 miss. A low L2 hit-to- 
miss ratio increases the ayexage latency of an Ll miss because 

30 more LlMsses result in lcmger latency L2 misses. Thus, a Ll 
cache miss could be disabled as a thread switch event if the 
executing code has a higji L2 hit- to -miss ratio because the L2 
cache data access latency is less than the thread switch 
penalty. A Ll cache miss would be enabled as a thread switch 

35 event when executing software code with a low L2 hit-to-miss 
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ratio because the LI cache miss is ^likely to turn into a longer 
latency L2 cachd miss. 1 ' - 

Some types of latency events are not readily detectable. 
For instance/ in some systems the L2 cache outputs a signal to 
""the instruction unit when a cache miss occurs. Other 1*2 
caches, however, do^ hot output such- a signal, as in for 
example, if the L2 cache controller were on a separate chip 
from the processor' and accordingly, the r -prociessor :,i cannot 
readily determine a state change, in these ar<Aitectures, the 
processor can include a cycle counter for each outstanding LI 
cache miss. If the miss data has not been returned from the 
L2 cache after a predetermined number of cycles, the processor 
acts as if there had been a L2 cache miss and changes the 
thread's state accordingly. This algorithm is also applicable 
to other cases where there are more than one distinct type of 
latiency. As an example only, : for a L2 cache miss in a 
multiprocessor, the latency of data from main memory may be 
significantly different than the latency of data from another 
processor. These two events may be assigned different states 
in the thread state register. If no signal exists to 
distinguish the states \ a counter may be used to estimate which 
state the thread should' be in ^f ter" it encounters a L2 cache 
miss 1 

The thread switch control register 410 is a software 
programmable register which selects the events to generate 
thread switching and hasa separate enable bit for each defined 
thread switch event. Although the embodiment described herein 
does not implement a separate r thread switch control register 
410 for each thread, separate thread switch control registers 
410 for each thrfead could be implement ed to provide more 
flexibility arid performance at the cost of more hardware and 
complexity. 

The thread Switch contirbl register 410 can be written by 
a service processor with software such as a dynamic scan 
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communications interface disclosed in U.S. Patent No. 5,079,725 
entitled Chip ident&fl cation Method for Use with Scan Design 
systems and Scan Testing Techniques or by the processor itself 
with software system code. The contents of the thread switch 
control register 410 is used by the thread switch controller 
450 to enable or disable the generation of a thread switch. 
A value of one in the register 4i0 enables the event associated 
with that bit to' generate a thread switch. A value of zero in 
the thread switch control register 410 disables the event 
associated with that bit from generating a thread switch. Of 
courk£, an instruction in the executing thread could disable 
any or all of the thread switch conditions for that particular 
or for other threads. The following table shows the 
association between thread switch events and their enable bits 
in the register 410. 

Thread Switch Control Register Bit Assignment 

(0) Switch cm LI data cache fetch miss 

(1) Switch on LI data cache stoic miss 

(2) Switch on LI instruction cache miss 

(3) Switch on instruction TLB miss 

(4) Switch on L2 cache fetch miss 

(5) Switch on L2 cache store miss 

(6) Switch on L2 instruction cache miss 

(7) Switch on data TLB/segment lookaside buffer miss 

(8) Switch on L2 cache miss and dormant thread not L2 cache miss 

(9) Switch when thread switch time-out value reached 

(10) Switch when 12 cache data returned 

(11) . . > Switch on IO external accesses , . 

(12) Switch on double-X store: miss on first of two* 

(13) Switch on double-X store: miss oh second of two* 

(14) Switch on store rrndtiple/siring: miss on any access 

(15) Switch on load multiple/string: miss on any access 

(16) Reserved 

(17> Switch on double-X load:, miss on first of two* 

(18) Switch on double-X load: miss on second of two* 

(19) Switch on or 1,1,1 instnictian if machine state register (problem state) bit, 
msr(pr)=l. Allows (Software priority change independent of msrtjw). If 
bit 19 is one, or 1,14, instruction sets low priority. If bit 19 is zero, 
priority is set to low only if msr(pr)=0 when the or 1,1,1 instruction is 

....... exec ute d See changing, priority with software, to be discussed late, 

(20) Reserved 

(21) Thread switch priority enable :! r - 
(2229) Reserved 

(30*31) Forward progress count 

(32:63) Reserved in 64 bit register imrriementation 
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* A doubieOC load/store refers to loading or storing an elementary hattword, a 
word, or a doable ward, that crosses z ckniWeword boundary. A douhie-X 
loariVstore in this context is not a load or store of multiple words or a string of 
words. e 



Thread Switch Time -out Register . 

As discussed . above,,. . coarse grained multithreaded 
, ..processors rely on lpng latency, events to trigger thread 
switching. Sometimes during execution, , a processor in a 
multiprocessor environment or a background Jtihredd. in a 

10 multithreaded architecture, has ownership of. a resource that 
can have only a single owner and another processor or active 
thread requires access to the resource before it can make 
forward progress. Examples include updating a memory page 
table or obtaining a task f rom a task dispatcher. The 

15 inability of the active thread; to obtain ownership of the 
resource does not result in a thread switch event, nonetheless, 
the thread is spinning in.a loop unable to do useful work. In 
this case, the background thread that holds the resource does 
not obtain access to the processor so that it can free up the 

20 resource because it never encountered a thread switch event and 
does not become the active thread. 

Allocating processing cycles • among the threads is another 
concern; if software code running on a thread seldom encounters 
long latency switch events compared to software code running 

25 on the other threads in 7 the same processor, that thread will 

get more than it's fair share of processing cycles. Yet 
another excessive delay that may exceed the maximum acceptable 
time is the latency of an , inactive thread waiting to service 
an external interrupt within a limited period of time or some 

30 other event . external .'it&./thB^jfroQesBpr^ Thus, it becomes 
preferable to force a thread switch to the dormant thread after 
some time if no useful processing is being accomplished to 
prevent the system from hanging. 
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The logic to force a thread switch after a period of time 
is a thread- switch , time-out register, 430 (Figure 4), a 
decrementer, and a decrementer register to hold the decremented 
value* The thread switch time -out register 430 holds a, thread 
5 switch time-out value. The thread switch time-out register 430 

c -implementation used, .in, r this embodiment is shown in the 
following table: /f . A 

Thread Switch Time-out Register Bits 

.. - ; : r;.v(0^2))-, Reserved - 

10 (2231) Thread switch time-out vahic 

The embodiment of the invention described herein does not 
,* implement, a separate thread switch time-out register 430 for 
each thread, although that could be done to provide more 
flexibility. ^ Similarly, if there are multiple threads, each 

15 thread need not have the same thread switch time-out value. 

Each time a thread switch occurs, the thread switch time-out 
value from the thread switch time-out register 430 is loaded 
. by hardware into the decrement register. The decrement 
register is decremented once each cycle until the decrement 

20 register value equals zero, then a signal is sent to the thread 
switch controller 450 which forces a thread switch unless no 
other thread is ready to process instructions. For example, 
if all other threads in the system are waiting on a cache miss 
and are not ready to execute instructions, the thread switch 

25 controller 450 does not force a thread switch. If no other 
thread is ready to process ihstiructions when the value in the 
decrement register ..reaches zero,, the decremented value is 
frozen at ; zero until another, thread . is ready to process 
instructions, at which point a thread switch ocpurs and the 

30 decrement register is reloaded with a thread switch time-out 
value for that thread.. Similarly , the. decrement register could 
just as easily be named an. increment register and when a thread 
is executing the register, cpuld increment up to same 
predetermined value when a thread switch would be forced. 
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The thread switch time-out register 430 can be written by 
a service processor as described *ab6Ve or by the processor 
itself with software "code. The thread switch time-out value 
loaded into the tfiread switch time-out register 430 can be 
customized according to specif ic hardware configuration and/or 
specific software code' to minimize wasted cycles resulting from 
unnecessary thread switching. Too high of a value in the 
thread switch time-out register 430 can result in reduced 
performance when the active thread is waiting for a resource 
held by another thread or if response latency fbr;:;an external 
interrupt or some other event external to the processor is too 
long. Too high of a value can also prevent fairness if one 
thread experiences a high number of thread switch events and 
the other does not. A thread switch time -out value twice to 
several times longer than the most frequent longest latency 
event that causes a thread switch is recommended, e.g., access 
to main memory. Forcing a thread switch after waiting the 
number of cycles specified in the "thread switch time-out 
Register 430 prevents system hangs due to shared resource 
contention, enforces fairness of processor cycle allocation 
between threads, and limits the maximum response latency to 
external interrupts and other events external to the processor. 

Forward Progress Guarantee 

That at least one 1 instruction must be executed each time 
a thread switch Occurs arid a new thread becomes active is too 
restrictive in certain' circumstances , such as when a single 
instruction generates multiple cache accesses and/or multiple 
cache misses. For example, a fetch instruction may cause an 
LI I -cache 150 miss if the instruction requested is not in the 
cache; but When the "instruction returns , required data may not 
be available in the" LI D- cache 120. Likewise, a miss in 
translation lookaside buf£er 250 can also result in a data 
cache miss. So, if forward progress is strictly enforced, 
misses on subsequent accesses do not result in thread switches. 
A second problem is that some cache misses may require a large 
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number of cycles to complete, during which time another thread 
may experience a .^aq&g ,miss at the same cache level which cam 

. be completed in much less time. , If, when returning to the 
first thread, the strict forward progress is enforced, the 
processor is unable to switch to : the thread with the shorter 

. cache miss . % : _ , : : 

:g To remedy the , .problem of thrashing wherein each thread is 
locked in a j repetitive cycle^of switching threads without any 
iMtructions QXjQQuting, - there exists a forward progress count 
register 420 -(Figure 4) which allows up to a programmable 
maximum number of thread switches, called the forward progress 
threshold value. After that maximum number of thread switches, 
an instruction must be completed before switching can occur 
again, in this way, thrashing is prevented. Forward progress 
count register 420 may actually be bits 30:31 in the thread 
switch control register 410 or a software programmable forward 
progress threshold, register for the processor. The forward 
progress, .count? logic uses, bits 15:17 of the thread state 
registers 442;/- 444 that indicate the state of the threads and 
are allocated for the number of thread switches a thread has 
experienced without an instruction executing. Preferably, then 
these bits, comprise the forward progress counter.. 

When a thread changes state, invoking the thread switch 
algorithm, if at n least,, one ins truction has completed, in the 
active thread, the forward- progress counter for the active 
thread .is reset : and the thread, switch algorithm continues to 
compare thread, states between, the threads in the processor. 
If no instruction has completed , the forward -progress counter 
value, in the thread .state ; -regijatgr of . the active thread is 
compared to the, forward ; ; pr9greBg ^threshold value. If the 
counter value or state. is jiot equal to the threshold value, the 
thread switch algorithm. continues to evaluate the thread states 
of the threads in : the processor-.. \. Then : if a thread switch 
occurs, the forward- progress counter is incremented. If, 
however, the. counter value is equal to the threshold value, no 
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thread 1 switch will occur until -.ah- instruction can execute, 
i.e.* until forwaird progress occurs. Note that if the 
threshold register has value zero, at least one instruction 
must complete within the active thread before switching to 
another thread. If each thread switch requires three processor 
cycles and if there are two threads and if the thread -switch 
logic is programmed to stop trying to switch threads after five 
1 ^tiries ; then the maximum number cf cycles . that the processor 
will thrash is thirty cycles . * One cf skill in 'the art can 
appreciate that there a potential conflict exists- between 
prohibiting a thread switch because no forward progress will 
f be made on one hand and, on the other hand, forcing a thread 
switch because the time-out count has been exceeded. Such a 
° conflict can easily be resolved according to architecture and 
software. : 

Figure 6 is a flowchart of the forward progress count 
feature of thread switch logic 400 which prevents thrashing. 
At block 610, the forward progress counter bits 15:17 in thread 
state register 442 pertaining to thread TO are reset to state 
111. Execution of this thread is attempted in block 620 and 
the state changes to 000. ' If an -instruction successfully 
executes on thread TO, the state of thread TO returns to 111 
and remains so. If, however, thread TO cannot execute an 
instruction, a thread switch occurs to thread Tl, or another 
background thread if more than two threads are permitted in the 
processor architecture. When a thread switch occurs away from 
Tl or the other background thread - and execution returns to 
thread TO, a second attempt to execute thread TO occurs and the 
state of thread TO becomes 001 as in block 630. Again, if 
thread TO encounters a thread switch event, control of the 
processor is switched away from 1 thread TO to another thread. 
Similarly, whenever a thread switch occurs from the other 
thread, e.g., Tl, back to thread TO, the state of TO changes 
to 010 on this third attempt to execute TO (block 640) ; to Oil 
on the fourth attempt to execute TO (block 650) , and to state 
200 on the fifth attempt to execute TO (block 660). 
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In this implementation, there are five attempts to .switch 
to thread TO. After .the fifth attempt or whenever the value 
of bits 15:17 in the thread state ( register (TSR) ,442 is, equal 
to the value of bits 30:31 plus one in the thread switch 
control register (TSC) 410, ; i..e v whenever TSC{30:31) = TSR 
(15:17), no thread switch away from thread TO occurs . It will 

- be appreciated that f ive attempts is an arbitrary number; the 
maximum number of allowable switches with unsuccessful 
execution, i.e., the forward progress threshold value, is 
programmable and it may be realized in certain architectures 
that five is too many switches, and in other architectures,, 

; five is too, few. in any event, the relationship between the 
number of times that an attempt to switch to a thread with no 
instructions executing must be compared with a threshold value 
and once that threshold value has been reached, no thread 
switch occurs away from that thread and the processor waits 
until the latency associated, with that thread is resolved. In 
the embodiment (described herein, the state of the thread 
represented by bits 15:17 of the thread state register 442 is 
compared with bits .30*31 in the thread switch control register 
410. Special handling, r . for particular events that have 
extremely long latency, such as interaction with input/output 
devices, to prevent prematurely. blocking thread switching with 
forward progress logic improves processor performance. One way 
to handle these, extremely long, latency events is to. block the 
incrementing of the forward progress counter or ignore the 
output signal of , the comparison between the forward progress 
counter and the threshold value , if data has not returned. 
Another way to handle extremely long latency events is to use 
a separate larger forward progress count for these particular 
events . 

Thread Switch Manager . 

The thread. state for .^11 ..software threads dispatched to 
the processor is preferably., maintained in the thread state 
registers 442 and 444 of Figure 4 as described. In a single 
processor one thread executes its instructions at a time and 
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all other threads are dormant. Execution is switched from the 
active thread to a dormant thread- when the active thread 
encounters a long -latency event as discussed above with respect 
v to the forward progress register 420, the thread switch control 
register 410, or the thread switch time-out register 430. 
' Independent "of Which t&read\ts : active, these hardware registers 
Tise Conditions^ that do hot dynamically change during the course 

Flexibility to change thread switch *'cbndit:icm8-'^y r a 'thread 
switch manager improves overall system performance. A software 
thread switch manager can' alter the frequency of thread 

^switching, increase execution cycles' available for a critical 
task, ahd decrease the overall cycles lost because of thread 
switch latency. The thread switch manager can be programmed 
either at compile time or during execution by the operating 
system, e.g. , a locking lobp can change the frequency of thread 
switches; or an operating system tksk can be dispatched because 
a dormant thread in a lower priority statie is waiting for an 
external interrupt or is otherwise ready. It may be 
advantageous to disallow or decrease the frequency of thread 
switches away from an active thread so that perf ormance of the 
current instruction stream 1 does mot suffer the latencies 
resulting from switching into and out of iti Alternatively, 
a thread can forgo some or ail l 6f its execution cycles by 
essentially lowering itk' priority, and as a result, decrease 
the' frequency of switches into it or increase the frequency of 

* switches out of the * thread *to : enhance overall system 
performance. The thread switch manager may also 

unconditionally force or inhibit a thread switch, or influence 
which thread is next selected for execution. 

A multiple-priority thread switching scheme assigns a 
priority value to each thread to qualify the conditions that 
cause a switch. It may aLlso b6 desirable in some cases to have 
the hardware alter thread' priority. For instance, a low- 
priority thread may 'be waiting on some event, which when it 
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occurs, the hardware; can raise the priority of the thread to 
influence the response time of. the thread to the event. 
Relative priorities between threads or the priority of a 
certain thread will influence- the handling of such an event* 
The priorities of the threads can be adjusted by the thread 
switch manager through the use of one .or more instructions, or 
by hardware in response to an . event . The thread switch, manager 
^alters the ractions performed by the. hardware thread switch 
logic to effectively change the relative priority of the 
threads • 

Three priorities are used with the embodiment described 
herein of two threads and provides sufficient distinction 
between threads to allow, tuning of performance without 
adversely affecting system performance. With three priorities, 
two threads can^have an equal status of medium priority. The 
choice of three priorities for two threads is not intended to 
be limiting. * In r sqme architectures a "normal" state may be 
that one thread -always has a higher priority than the other 
threads. It , is -intended to be within the scope of the 
invention to cover more than two threads of execution having 
one or multiple priorities . that can be. set in hardware or 
programmed by software. 

The three priorities of each. thread are high, medium, and 
low. When the priority of thread TO is the same as thread T3L_, 
there is * no.-.-efrf ect on the thread switching logic. Both threads 
have equal priority so neither is given an execution time 
advantage, when the priority of thread TO is greater than the 
priority of : thread Tl, . thread, switching from TO to Tl is 
disabled for all LI cache-misses, i.e. data load, data store, 
and instruction fetch, .because £4 „ cache misses are resolved 
much faster than . pther, conditions such as L2 misses and 
translates . Thread TO is given a better chance of receiving 
more execution cycles than thread Tl which allows thread TO to 
continue execution so long as it does not waste an excessive 
number of execution cycles The, processor, however, will still 
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relinquish control to thread Tl- if thread TO experiences a 
relatively long execution latencyv -unread switching from Tl 
to TO is unaffected; except that a switch occurs when dormant 
* thread TO is ready in which case thread TO preempts thread Tl. 

This case would be expected to occur when thread TO switches 
c r away because of an 12 cache miss or translation request, and 
th€' condition is resolved in the background while thread TO is 
executing/ The case of thread 1 ra having a priority- Lefes than 
thread Tl is analogous to the case above, 1 with the thread 
designation revers ed . ^ 

There are different possible" approaches to inclement ing 
management of thread switching by changing thread priority. 
New instructions can be added to the processor architecture. 
Existing processor instructions having side effects that have 
the desired actions can also be used.- Several factors 
influence the choice among the methods of allowing software 
control: (a) the ease of redefining architecture to include 
new instructions and the effect of architecture changes on 
existing processors; (b) the desirability of running identical 
software on different versions of processors; (c) the 
performance tradeoffs between using new; special purpose 
instructions versus reusing existing instructions and defining 
resultant side effects; (d) the desired level of control by the 
software, e.g., whether the effect can be caused by every 
execution of some existing : ins t ruction V su6h as a specific load 
or store, or whether more control is- needed, by adding an 
instruction to the strekra to specifically cause the effect. 

The architecture described herein preferably takes 
advantage of an unused instruction whose values do not change 
the architected generail' purpose registers of the processor; 
this feature is critical for retrofitting multithreading 
capabilities into a processor architecture, otherwise special 
instructions clan be cbded\ f The instruction is a preferred 
nop" or 6,0,0; other instructions, however, can effectively act 
as a nop. By using different versions of the or instruction, 
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or 0 f 0,0 or 1,2,1 etc. to alter thread priority, the same 
instruction stream may execute on a processor without adverse 
effects such as illegal instruction interrupts. An extension 
uses the state of the machine state register to alter the 
5. , meaning of these instructions . - For example, it > may be 
undesirable to allow a user to code some or all of these, thread 
apriority instructions and access, the functions they provide. 
, \ The special functions they provide jnnay be, defined to occur only 
in certain modes of execution, they will have no effect in 
10 other, modes and will -be executed normally, as a nop. 

One : possible implementation, using a dual -thread 

, multithreaded processor, uses three instructions which become 

part of the executing software itself to change the priority 

of itself: : : 

15 tsop 1 or 1,1,1 , - Switch to dormant thread . 4 . 

tsop 2 or i,/,/ - Set active thread to LOW priority 

- Switch to dormant thread ' 

- NOrrg: Only valid in privOeged mode unless TSC[1?]»1 
tsop 3 or 2X2 - Set active thread to MEDIUM priority 

20 tsop 4 or J&$ - Set active thread to HK3H priority 

r NOTE: ,Oriy valid in privileged mode 

instructions tsop 1 arid tsop 2 can be the same instruction 
as embodied herein as or 1/1 ,1 but they can also be separate 
instructions. These instructions interact with bits 19 and 21 

25 of the thread switch control register 410 and the 
problem/privilege bit of the "machine state register as 
described herein. * If bit 21 of the thread switch control 
register 410 has a value of otie, tie thread switch manager cam 
set the priority of its thread to one of three priorities 

30 represented in the thread state register at bits 18:19. If bit 
19 of the thread switch control register 410 has a value zero, 
then the instruction tsop 2 thread switch and thread priority 
setting is controlled by'the problem/privilege bit of the 
machine state register. On the other hand, if bit 19 of the 

35 thread switch control register 410 has a value one, or if the 

problem/privilege bit of the machine state register has a value 
zero and the instruction or 1/14 is present in the code, the 
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priority for the active tliread is J set to low and execution is 
immediately switched to the dormant 6r A background thread if the 
dormant thread is enabled. The instruction or 2,2,2 sets the 
priority of the active thread to medium regardless of the value 
of the problem/ptivilege bit of the machine state register* 
r And the instruction or 3,3*3/- when the problem/privilege bit 
of the machine state register bit hate a value of zero, sets the 
Apriority of the active thread to tiighi If -bit 21 of tm thread 
switch control register 32C is zero, fchfe priority for both 
threads is set to medium and the effect of 1 the v or x,x,x 
instructions on the priority is blocked. If an external 
interrupt request is active; arid if the corresponding thread's 
priority is low, 1 that thread's priority is set to medium. 

The events altered by the thread priorities are: (1) 
switch on LI D- cache miss to Iq&d data; {2) switch on Ll D- 
cache miss for storing data; (3) switch on Ll I-cache miss on 
an instruction fetch; and (4) switch if the dormant thread in 
ready state. In addition, external interrupt activation may 
alter the corresponding thread' s* priority. The following table 
shows the effect of priority on -conditions that cause a thread 
switch. A simple TSC entry in columns three and four means to 
use the conditions set forth in. the .thread switch control- (TSC) 
register 410 to initiate a thread switch. An entry of TSC [0:2] 
treated as 0 means that bits 0:2 of the thread switch control 
register 410 are treated as if the value of those bits are zero 
for that thread and the other. bits in the thread switch control 
register 410 are used as is for defining the conditions that 
cause thread switches. The ; phrase when thread TO ready in 
column four means . that a switch to thread TO occurs as soon as 
thread TO .is no longer waiting on the miss event that caused 
it to be switched : put The -phrase when thread Tl ready in 
column 3 means that, a switch to thread Tl occurs as soon as 
thread Tl is no longer waiting on the miss event that caused 
it to be switched out. If the.miss event is a thread switch 
time-out, there is no guarantee thjat the lower priority thread 
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completes an instruction before the higher priority thread 
switches back in. 



| TO Priority 


77 Priority . 


TO Thread Switch 
Conditions 


77 Thread Switch 
Ccmdittons 


I High 


High 


TSC 


TSC 


i High • 


Medium ~ - 


TSC(0:2) mated as 0 


TSC or if TO ready 


| High 


Low 


TSC[fc2] treated as 0 


TSC or if TO ready 


[I Medium : - 


High 


TSGorif 77 ready 


TSC[0:2] treated as 0 


| Medium - ^ - 


Medium' * 


TSC 


TSC 


| Medium 


Low 


TSC[0:2) treated as 0 


TSC or if TV ready 


Low 


High 


TSC or if 77 reacty 


TSC(0^] treated as 0 


Low 


Medium 


TSC or if 77 ready 


TSC[0:2] treated as 0 


Low 


Low 


TSC 


TSC 



It is recommended that a thread doing no productive work 
be glyen Jow priority to avoid a loss in performance even if 
every instruction in the idle loop causes a thread switch. 
Yet/ it is still important to allow hardware to alter thread 
priority if an external interrupt is requested to a thread set 
at low priority. In this case the thread is raised to medium 
priority, to allow a quicker response to the interrupt . This 
allows a thread waiting pa an^external event to set itself at 
low priority, where .it will, stay until the event is signalled. 

While the " invention has been, described in connection with 
what is presently considered, the most practical and preferred 
embodiments, it is to be understood that the invention is not 
limited to the disclosed embodiments, but on the contrary, is 
intended to cover various modifications and equivalent 
arrangements included within the spirit and scope of the 
appended claims. 
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Claims - 

A method of computer processing, ; comprising the steps of: 

(a) setting a forward progress threshold value; 

(b) counting a -number cf times that execution has been 
switched, to a first thread without an instruction of 
the: first, thread being executed in a multithreaded 

;;' : "; process or; -* _ v ?■ J 

(c) not switching execution to a ; second thread; if the 
number of times that execution has been ri switched to 
the first thread is equal to, the forward " progress 
threshold value and waiting until the' first thread 
is able to execute. 

2 . The method of blaim 1 wherein the forward progress 
threshold value is programmable by code. 

3 . The method of Claim 1 or 2 wherein the forward progress 
threshold value is stored in a forward progress count 
register. 

4. The method of any one of Claims 1 to 3 wherein the step 
of counting a number of times that execution has been 

' switched to the first thread further comprises changing 
*'a state of the first thread in a first state register. 

5. The method of any one of Claims 1 to 4 wherein a latency 
event inhibits executioh of an instruction in the first 
thread. 
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1 6. A method of computer processing, comprising the steps of: 

2 (a) setting a forward progress threshold value in a 

3 forward progress count ..register; 

4 (b) resetting a state qf a first thread in a first 

5 thread state register; 

6 ; t (c) attempting execution, of the first thread in a 

7 multithreaded processor; 

8 v - t (d) changing , the ste^te of the first thread in the first 

9 - :j . -thread- state register; 

10 ;(e) v if the first, thread „ is able to execute an 

11 : - . instruction, then . executing an instruction of the 

12 . f i^rst thread and going to step (b) ; 

13 (f) if the first thread is unable to execute, 

14 determining if the state of the first thread is 

15 equal to a forward progress threshold value, if not, 

16 going to step (g) ; otherwise going to step (j); 

17 (g) switching execution to a second thread in the 

18 . multithreaded processor; 

19 (h) continuing execution of the second thread until the 

20 second thread is unable to execute; 

21 . (i) switching execution to the first thread in the 

22 multithreaded processor and returning to step (b) j 

23 , (j) refraining from switching execution to the second 

24 . .thread until first thread is able to execute an 

25 . instruction to .guarantee forward progress in the 

26 multithreaded processor haying at least two threads 
27 \ of execution. 

1 7. A processor comprising a multithreaded processor (100) 

2 ; wherein a thread switch is inhibited because an 

3 . ... instruction of a thread. does not execute. 
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A computer processor, comprising: 

(a) a multithreaded processor (100) to switch processing 
between at least two threads of instructions; 

(b) a forward progress count register {420} operatively 
connected to the multithreaded processor and having 
a first forward' progress threshold value; 

(c) at least one thread state register (440) operatively 
connected to the forward progress count register and 
to the multithreaded processor,- to store a state of 
the threads of instructions wherein the state of 
each thread of instructions changes when the 
processor switches processing to each thread; 

wherein when the state of a thread is equal to the first 
forward progress threshold value, the multithreaded 
processor does not switch threads but waits for resolution 
bf at least one latency event. 



1 9. A computer system (10)/ comprising: 

2 (a) a multithreaded processor (100) capable of switching 

3 processing 1 between >: at least two threads of 

4 instructions when the" multithreaded processor 

5 experiences one of a plurality of processor latency 

6 ' ; events; 

7 (b) a forward progress count register (420) having at 

8 least one foi^ard jprbgresS threshold value which is 

9 operatively connected to the multithreaded 

10 processor; 

11 (c) at least one thread state register (440) operatively 

12 connected to the forward progress count register and 

13 to the rnulti threaded processor; to store a state of 

14 the threads J "bf" -instruct ions wherein the state of 

15 eacfi thread ! ' of ; * instructions changes when the 

16 processor switches processing to each thread; 

17 (d) a plurality of internal connections connecting the 

18 multithreaded processor to a plurality of memory 

19 elements (120, 130, 140, 150) wherein access to any 

20 of the plurality of memory elements by the 
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multithreaded processor causes an internal processor 
la t encyo e^enft ; 

wherein . when one of the , threads executing in the 
multithreaded processor is,. unable to continue execution 
because of one of the : prpcesspr- latency; events and the 
multithreaded processor- switches > e«^cutipn tc* another of 
,>,. .the threads, and changes the stajte of both, threads, the 
x.uq —state of both ; threads : are pompared ,/with at least one 
X Qr*rard jprpgress threshold value* and when the state of 
- . a ^p^r titular thread r is equal to at least one forward 
progress threshold value, the multithreaded processor does 
not switch away from the particular thread. 

10. The computer system of Claim 3 further comprising at least 
one external connection (155) connecting the multithreaded 
processor to at least one external memory device (170), 
at least one external qorarminication device (174) , an 
external* computer network (176), or at least one 
input /output device (178) wherein access to any of the 
devices or the network by the multithreaded processor 
causes an external processor latency event. 

11. The computer system of Claim 9 or 10 further comprising 
an instruction to block the change of state of the 
particular thread in the thread state register. 

12. The computer system of any one of Claims 9 to 11 wherein 
an occurrence of an external processor latency event 
blocks the change of state of the particular thread in the 
thread state register. 

13. The computer system of any one of Claims 9 to 12 wherein 
regardless of the state of the particular thread, the 
multithreaded processor switches away from the particular 
thread. 
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1 : l '- v 14. The computer system of any one of Claims 9 to 13 further 

2 comprising a first forward progress threshold value 

3 corresponding to an internal processor latency event and 

4 a second forward progress threshold value corresponding 

5 to ah external processor latency event wherein the 

6 : * nail t*i threaded processed - does not switch from a first 

7 < thread if the first thread is experiencing an internal 

8 processor latency event and ' the first forward progress 

9 threshold value equals the state of the first thread or 

10 wherein when the multithreaded processor does not switch 

11 from a second thread if the second thread is experiencing 

12 an external processor latency evant and the second forward 

13 progress threshold value equals the state of the second 

14 thread, depending upon which thread has a higher priority. 

1 15 • A computer processor; comprising: > 

2 (a) means for establishing a threshold value; 

3 (b) means for tracking a number of thread switches in a 

4 multithreaded processor; 

5 (c) means for inhibiting a thread switch when the number 

6 of thread switches equals the threshold value. 
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